Rewrite benchmarking scripts with Python. #5082
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GMNGeoffrey
reviewed
Mar 12, 2021
This causes a lot of issues....
antiagainst
approved these changes
Mar 15, 2021
GMNGeoffrey
approved these changes
Mar 15, 2021
|
|
||
| import configuration | ||
|
|
||
| PATTERN = re.compile(r"BM_(\w+)(.+)/real_time(\s+) (?P<ms>[.0-9]+) ms") |
There was a problem hiding this comment.
I still think we should be using the more structured output instead of parsing the human-readable output... but this is ok for now
There was a problem hiding this comment.
Yes, I think we can let it report a csv or something. I keep this because this is the same behavior as before. I will consider this.
Co-authored-by: Geoffrey Martin-Noble <[email protected]>
Merged
copybara-service bot
pushed a commit
that referenced
this pull request
Mar 16, 2021
* 9ae2e49 Rename compiler2 to compiler (#5113) * 4168916 [CodeGen] Add a pattern to canonicalize HAL interface load/store (#5103) * 5b6e7d1 Rewrite benchmarking scripts with Python. (#5082) * 36912bc Merge google -> main (#5117) * ec9f244 Merge branch 'main' into google-to-main * e479591 Disable native-py check from buildifier action * 02bb838 Enable concat, gather and torch_index_select on Linalg on tensors path. (#5053) * 31c71df Integrate MLIR-EmitC at iml130/mlir-emitc@3483f16 (#5114) * 2839b02 Fix missing <cstddef> header file. (#5112) * 5670121 Fixing flow.dispatch.workgroups tied operands when removing args. (#5108) COPYBARA_INTEGRATE_REVIEW=#5125 from not-jenni:main-to-google 4936e55 PiperOrigin-RevId: 363244672
iree-github-actions-bot
pushed a commit
that referenced
this pull request
Mar 16, 2021
* 9ae2e49 Rename compiler2 to compiler (#5113) * 4168916 [CodeGen] Add a pattern to canonicalize HAL interface load/store (#5103) * 5b6e7d1 Rewrite benchmarking scripts with Python. (#5082) * 36912bc Merge google -> main (#5117) * ec9f244 Merge branch 'main' into google-to-main * e479591 Disable native-py check from buildifier action * 02bb838 Enable concat, gather and torch_index_select on Linalg on tensors path. (#5053) * 31c71df Integrate MLIR-EmitC at iml130/mlir-emitc@3483f16 (#5114) * 2839b02 Fix missing <cstddef> header file. (#5112) * 5670121 Fixing flow.dispatch.workgroups tied operands when removing args. (#5108) PiperOrigin-RevId: 363244672
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This moves the config to utils.py, which makes the compilation script and benchmarking script share the same config. A ModelBenchmarkInfo contains a phone list. This allows us to set different config for different models. This is useful for doing batch mode benchmarking.